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ABSTRACT 

Dealing  with  an  ever  increasing  amount  of  data  is  a  challenge  that  military  intelligence  analysts  or  team  of  analysts  face 
day  to  day.  Increased  individual  and  collective  comprehension  goes  through  collaboration  between  people.  Better  is  the 
collaboration,  better  will  be  the  comprehension.  Nowadays,  various  technologies  support  and  enhance  collaboration  by 
allowing  people  to  connect  and  collaborate  in  settings  as  varied  as  across  mobile  devices,  over  networked  computers, 
display  walls,  tabletop  surfaces,  to  name  just  a  few.  A  powerful  collaboration  system  includes  traditional  and  multi¬ 
modal  visualization  features  to  achieve  effective  human  communication.  Interactive  visualization  strengthens 
collaboration  because  this  approach  is  conducive  to  incrementally  building  a  mental  assessment  of  the  data  meaning.  The 
purpose  of  this  paper  is  to  present  an  overview  of  the  envisioned  collaboration  architecture  and  the  interactive 
visualization  concepts  underlying  the  Sensemaking  Support  System  prototype  developed  to  support  analysts  in  the 
context  of  the  Joint  Intelligence  Collection  and  Analysis  Capability  project  at  DRDC  Valcartier.  It  presents  the  current 
version  of  the  architecture,  discusses  future  capabilities  to  help  analyst(s)  in  the  accomplishment  of  their  tasks  and 
finally  recommends  collaboration  and  visualization  technologies  allowing  to  go  a  step  further  both  as  individual  and  as  a 
team. 
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1.  INTRODUCTION 

Dealing  with  an  ever  increasing  range  of  data  of  diverse  types  is  a  challenge  that  (military  intelligence)  analysts  face  day 
to  day.  Digital  data  is  regularly  accessed  by  these  people  in  order  to  visualize,  interpret  and  analyze  information,  to 
comprehend  a  complex  situation  either  individually  or  as  a  team,  to  share  results  with  other  (team)  members,  to  fuse 
information  from  different  sources  of  data,  or  to  support  decisions  and  hypotheses.  Increasing  individual  and  collective 
comprehension  goes  through  collaboration  between  (co-localized  and/or  geographically  dispersed)  people.  We  work 
under  the  hypothesis  that  better  is  the  collaboration,  better  will  be  the  comprehension. 

Joint  Intelligence  Collection  and  Analysis  Capability  (JICAC),  the  R&D  project  in  which  the  work  presented  in  this 
paper  is  realized,  has  to  develop  and  demonstrate  an  advanced  prototype  of  an  Intelligence  production  capability 
intended  for  all-source,  multi-intelligence  analytical  teams  in  production  environments  of  the  future.  This  project  is  the 
first  instantiation  of  the  Future  Intelligence  Analysis  Capability  (FIAC)  vision  that  aims  at  advancing  the  state-of-the-art 
of  intelligence  methods  and  resources  in  order  to  adapt  to  and  anticipate  the  new  challenges  of  the  Canadian  Armed 
Forces  and  other  government  agencies.  As  expressed  in  (Poussart,  2013),  “FIAC  addresses  information  services:  (1) 
from  a  multi-intelligence  and  all- sources  perspective,  (2)  over  the  full  intelligence  cycle,  (3)  but  especially  toward 
enhancing  analysis  capabilities.  FIAC  seeks,  in  particular,  to  exploit  an  optimized  level  of  synergy  between  human 
cognition  and  machine  intelligence,  using  advanced,  collaborative  interaction.  A  key  paradigm  is  that  all  participants 
share  a  single,  unified  data/information  space,  irrespective  of  their  actual  physical  location.”  Figure  1  depicts  the  FIAC 
operating  concept.  The  bottom  pane  shows  the  conceptual  mapping  of  the  analysis  phases  of  the  intelligence  cycle,  based 
on  the  framework  introduced  in  (Pirolli  and  Card,  2005).  The  top  frame  provides  an  artist’s  rendition  of  an  analysts  team 
using  advanced  visualization  and  interaction  technologies,  virtual  reality,  spatial  computing,  virtual  assistants  that  are 
capable  of  operating  at  high  cognitive  levels,  extensible  work  spaces,  conferencing  with  remote  facilities,  etc.  The  side 
panes  represent  high  performance  computing  resources  for  intelligent  processing:  on  one  side  are  those  in  direct,  real 
time  support  for  the  on-going  analysis  work,  on  the  other  side  are  those  involved  in  the  optimized  management  of  the 
resources  (human  and  machine  based),  as  well  as  essential  infrastructure  for  communication,  archiving,  and  resource 
allocations. 

This  paper  presents  the  latest  architecture  version  underlying  the  Sensemaking  Support  System  (S3)  prototype  developed 
for  the  JICAC  project.  The  S3  is  a  step  towards  the  implementation  of  the  FIAC  concepts.  It  consists  of  various 
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intelligence  analysis  web  services  build  of  top  of  big  data  technologies  exploited  through  a  single  web  application.  The 
system  has  been  developed  following  main  design  principles  from  the  Service  Oriented  Architecture  -  Reference 
Architecture1  (SOA-RA)  technical  standard  produced  by  The  Open  Group.  Following  the  architecture  overview,  the 
paper  focusses  on  new  capabilities  of  interest  for  the  military  intelligence  analyst  of  the  future  and  pursues  with 
collaboration  and  visualization  tools  that  analyst  or  an  analysts  team  could  use  to  exploit  capabilities  included  in  the  S3. 
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Figure  1.  FIAC  operating  concept  (Poussart,  2013) 


2.  COLLABORATIVE  INTELLIGENCE  PRODUCTION  CAPABILITY 

The  proposed  collaborative  military  intelligence  production  capability  combines  the  current  S3  prototype  with  a  context 
management  sub-system  in  order  to  provide  adaptive  collaboration  features  in  support  to  the  whole  intelligence 
production  workflow.  The  resulting  context-aware  system  adapts  its  behavior  and  features  dynamically  according  to 
various  aspects  of  the  context  (e.g.  current  users,  their  roles,  tasks,  permissions  and  locations,  limited  nearby  resources, 
missions  objectives  and  so  on)  in  order  to  optimize  exploitation  of  the  system  while  providing  continual  system  feedback 
and  a  seamless  user  experience  across  the  whole  system. 

From  the  point  of  view  of  the  intelligence  analyst,  the  proposed  capability  provides  at  the  right  time  the  relevant  tools 
and  just  the  appropriate  information  elements  to  support  his/her  current  work  and  the  next  steps  while  avoiding  cognitive 
overload.  The  analyst  can  request  information  and  produce  results  while  the  system  is  responsible  for  delivering  the 
appropriate  notifications  to  concerned  assets.  On  the  other  hand,  the  system  has  to  balance  resource  allocation  and  data 
sharing  in  order  to  process  and  coordinate  various  requests  from  multiple  collaborators  while  ensuring  the  convergence 
of  the  whole  team  towards  a  common  objective. 


1  http://www.opengroup.org/soa/source-book/soa  refarch/ 


Figure  2  shows  an  overview  of  the  envisioned  architecture  integrating  the  context- adaptive  collaboration  capability  with 
current  and  future  components  of  the  S3  prototype  such  as  the  Unified  Data  Space  (UDS),  various  Intelligence 
Production  Services  and  a  “publish  &  subscribe”  Messaging  Service.  The  diagram  also  depicts  basic  data  flows  and 
interactions  with  the  users. 
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Figure  2.  Architectural  overview  of  the  proposed  collaborative  intelligence  production  capability 

S3  Components 

The  whole  S3  prototype  sits  on  the  UDS  where  validated  common  knowledge  is  stored.  Based  on  the  Lumify  software2, 
this  important  component  exploits  big  data  technologies  such  as  Hadoop3  and  Storm4  to  extract,  ingest,  process,  filter 
and  index  large  amounts  of  data  according  to  requests  and  operations  executed  from  the  S3  interfaces. 

Above  this  database  resides  the  Big  Data  Manager  responsible  for  transparent  data  transmission  between  the  UDS  and 
the  rest  of  the  S3  system. 

Another  core  component  is  the  Messaging  Service ,  realizing  some  of  the  needed  features  in  the  integration  layer  defined 
in  the  SOA-RA.  This  component  is  also  a  prerequisite  for  most  of  the  collaboration  and  interaction  features  mentioned 
further.  It  manages  multiple  independent  pipelines  for  partitioned  message  exchanges  across  a  shared  network  allowing 
multiple  services,  users  and  other  system  components  to  share  data  and  send  notifications  between  each  other  while 
maintaining  privacy  and  partitioning  between  the  various  topics. 


2  https://www.altamiracorp.com/blog/products/lumify 

3  http://hadoop.apache.org/ 

4  http://hortonworks.com/hadoop/storm/ 


Context  Manager 

The  proposed  Context  Manager  takes  care  of  everything  related  to  system  usage  and  current  status  of  the  situation.  It 
monitors  current  online  users,  available  services  and  devices;  it  keeps  sets  of  analysis  data  up-to-date;  it  manages  data 
and  notification  exchanges  between  users  and  services;  it  responds  and  adapts  to  situation  changes,  to  new  requests  and 
updated  targeting  priorities  while  providing  a  seamless  user  experience  and  proper  feedback  through  Interactive 
Visualization  Widgets. 

To  achieve  all  those  features,  the  Context  Manager  relies  on  multiple  sub-components,  each  responsible  to  store  and 
react  to  specific  aspects  of  the  context: 

•  The  Context  Model  is  the  actual  placeholder  for  persisted  contextual  information.  Context  changes  are 
transmitted  between  components  of  the  system  through  the  Messaging  Service. 

•  The  Analysis  Set  Manager  is  responsible  to  gather  from  the  UDS  and  keep  up-to-date  a  reference  set  of  the 
necessary  data  that  is  required  to  fulfill  the  information  needs  for  the  different  workspaces  currently  in  use. 

•  The  Workspace  Manager  holds  multiple  independent  workspaces  allowing  users  to  manipulate  datasets 
obtained  through  the  Analysis  Set  Manager  in  a  private  or  shared  data  space  before  publishing  results.  It  also 
allows  an  analyst  to  work  on  multiple  intelligence  problems  separately. 

•  The  Workflow  Orchestrator  exploits  contextual  information  to  plan  and  coordinate  the  concurrent  executions  of 
various  processing  and  collaboration  tasks  to  fulfill  requests  and  support  the  work  of  multiple  users  sharing 
limited  resources. 

•  The  Workflow  Tracer  keeps  tracks  of  every  step  followed  by  the  analyst  and  every  action  initiated  by  the 
Workflow  Orchestrator  during  his/her  work.  A  Workflow  Exploration  interface  is  used  to  keep  the  user  aware  of 
the  current  status  of  his/her  work,  to  backtrack  on  previous  actions,  or  simply  for  review. 

The  following  section  describes  in  more  details  the  contextual  information  managed  in  the  S3  system  and  the 
exploitation  of  the  Context  Manager  to  provide  adaptive  collaboration  interfaces. 

3.  CONTEXT  AWARENESS 

Taking  into  account  the  usage  context  has  considerable  implications  for  the  system.  Context  is  by  nature  a  concept  that  is 
not  easily  defined  and  context  modeling  can  involve  an  almost  infinite  number  of  elements.  One  has  to  draw  a  line  and 
decide  what  will  be  considered  depending  on  the  application.  It  can  be  constituted  of  a  number  of  relevant  pieces  of 
information  including  static  and  dynamic  elements  that  make  up  the  analyst’s  working  situation. 

Consequently,  only  some  aspects  are  to  be  considered  in  the  Context  Manager.  Here  is  a  non-exhaustive  list  of  context 
properties  divided  into  four  main  categories: 

•  Operational  context :  role,  rank,  current  task,  mission  objectives,  assigned  information  requests  and  tasks,  focus 
of  interest  (geographical  areas,  topics,  targets  and  so  on); 

•  User  context:  properties  and  preferences  of  each  single  user,  his/her  personality,  cognitive  style,  physical 
disabilities,  spoken  languages,  cultural  differences,  level  and  field(s)  of  expertise  and  so  on; 

•  Resources  and  collaboration  context:  information  space,  available  services  and  computing  power,  display  count 
and  sizes,  available  interaction  modes,  taskable  assets,  human  team  members  who  can  contribute  and  their  areas 
of  expertise;  and 

•  Extrinsic  context:  connectivity,  bandwidth,  geographical  location,  time  of  the  day,  weather,  working  conditions, 
and  other  characteristics  of  the  physical  working  environment  (mostly  elements  that  are  partially  or  totally  out- 
of-control  for  the  users). 

These  context  properties  are  stored  in  the  Context  Model.  The  Workflow  Orchestrator  exploits  every  possible  pieces  of 
contextual  information  to  ensure  an  optimized  execution  of  the  workflow.  The  Workspace  Manager  component  is 
mostly  related  to  the  user  context  since  it  keeps  track  of  the  data  the  user  is  working  on. 


For  a  visual  interface  to  support  the  user  in  an  optimal  fashion,  information  presentation  should  adapt  to  the  user’s 
working  context.  This  is  the  basis  of  intelligent  adaptive  interfaces.  There  are  three  aspects  of  the  user  experience  that 
can  be  adapted: 

•  Information  content:  choose  what  is  being  presented  to  the  user; 

•  Information  presentation:  adapt  how  the  data  is  being  presented;  and 

•  Proposed  actions:  produce  a  set  of  proposed  next  steps  for  the  task  currently  being  performed. 

Unless  the  use  cases  are  well  defined  and  mostly  static,  current  and  past  contextual  information  must  be  considered  by 
the  system  in  order  to  propose  appropriate  next  actions  to  users.  By  exploiting  relevant  aspects  of  the  context  such  as 
user  profiles,  past  actions,  availability  of  resources  and  so  on,  the  system  learns  from  usage  history  to  enhance  its 
prediction  and  profiling  capabilities  in  support  to  each  individual  user. 

The  next  section  covers  more  deeply  the  Workflow  Orchestrator  and  Workflow  Tracer  concepts  applied  to  track  and 
exploit  the  analyst’s  actions  while  performing  work  in  order  to  propose  next  steps  and  optimize  his/her  interactions  with 
the  other  team  members. 


4.  ANALYST’S  WORKFLOW 

In  the  context  of  military  intelligence  production,  it  is  unfortunately  common  that  military  personnel  are  assigned  tasks 
for  which  they  could  not  get  full  training  prior  to  the  assignment  or  do  not  perform  on  a  daily  basis.  There  is  an 
expectation  that  the  interface  should  support  the  workflow  of  those  users  by  assisting  them  in  real-time  during  the 
ongoing  process,  and  even  suggesting  a  sequence  of  steps  to  be  performed  next.  The  tools  designed  for  this  purpose  are 
the  Workflow  Tracer  and  the  Workflow  Orchestrator  components  which  are  part  of  the  proposed  Context  Manager 
capability. 

Workflow  Tracer 

For  some  well  understood  and  routine  processes  the  steps  sequence  can  be  predefined.  However,  there  are  a  number  of 
activities  where  the  steps  are  not  always  identical  and  sequential.  Intelligence  analysis  is  one  such  process.  Some 
sequences  of  actions  can  be  performed  in  multiple  orders  while  still  producing  similar  results.  Some  steps  could  be 
redundant,  recursive  or  conditional  to  some  prerequisites.  While  performing  an  exploratory  analysis,  one  could  also  wish 
to  step  back  to  a  previous  operation  to  try  a  different  approach  and  compare  the  results  obtained. 

The  Workflow  Tracer  records  every  step  and  constitutes  an  historic  of  previous  actions.  The  user  can  navigate  within  the 
traced  workflow  through  a  Workflow  Exploration  interface  and  create  branches  for  alternatives  approaches.  Intelligence 
analysts  are  often  performing  multiple  tasks  in  parallel  while  they  wait  for  new  information  to  arrive  or  when  a  particular 
assignment  sees  their  priority  increased.  In  the  Workflow  Tracer  and  its  associated  exploration  interface,  this  sequence  of 
recorded  tools  interactions  can  be  split  intelligently  along  working  lines  that  are  associated  with  the  different  high  level 
tasks  that  a  user  is  performing  in  parallel,  allowing  him/her  to  alternate  between  them. 

Recording  every  action  that  the  user  takes  with  the  Workflow  Exploration  interface  is  the  first  step  toward  building  a 
capability  for  analyzing  these  workflows  in  order  to  produce  useful  tasks  recommendations.  Keeping  a  history  of 
interactions  is  also  desirable  because  it  supports  audit  and  acts  as  a  reminder  to  the  user  for  what  has  been  done  in  the 
past  regarding  a  specific  question  or  dataset. 

Workflow  Orchestrator 

The  Workflow  Orchestrator  exploits  elements  of  the  context  to  plan  and  coordinate  the  execution  of  multiple 
Intelligence  Production  Services  in  order  to  achieve  the  processing  tasks  defined  by  the  analyst.  This  component  has  to 
handle  many  aspects  of  the  context  in  order  to  maintain  the  proper  execution  of  multiple  workflows  simultaneously  with 
limited  dynamic  resources.  Many  factors  and  events  can  affect  the  workflow  execution.  Some  are  desired  and  voluntary, 
some  are  unpredictable.  The  impact  and  the  response  also  vary  depending  on  the  situation.  Results,  notifications  and 
information  requests  are  displayed  through  the  Interactive  Visualization  Widgets. 

Within  the  Workflow  Orchestrator ,  the  user  can  select  the  next  actions  and  tools  that  will  be  used  on  the  data  or  as  part 
of  the  analysis  process.  This  selection  can  be  made  from  a  library  of  available  processing  steps.  This  can  be  very 
efficient  for  performing  repetitive  task.  At  any  time,  the  user  can  aggregate  a  group  of  processing  steps  to  form  a  higher 


level  processing  box  in  the  Workflow  Orchestrator.  This  new  box  is  then  made  available  in  the  processing  steps  library. 
This  allows  an  experimented  analyst  to  create  a  plan  that  can  later  be  followed  by  more  junior  personnel.  After  the 
higher  level  process  is  inserted  into  the  analyst’s  work  line,  each  of  the  steps  can  be  followed  in  a  continuous  flow.  The 
work  line  visualization  then  serves  as  reference  tool,  showing  which  step  the  user  is  now  performing  and  what  will  be 
the  next  one. 

The  Workflow  Orchestrator  also  goes  one  step  further  than  the  Workflow  Tracer  by  exploiting  recorded  usage 
information  to  provide  future  actions  planning  and  predictions  capabilities.  When  planning  the  next  step,  available 
actions  to  be  performed  by  the  analyst  could  be  recommended  based  on  what  the  system  learns  from  the  users.  Using 
pattern  recognition  on  sequences  of  processing  steps  forming  a  work  line  in  the  Workflow  Tracer  could  lead  to  the 
identification  of  groups  of  steps  that  are  often  performed  together  in  a  specific  order.  The  work  lines  performed  by  other 
analysts  can  be  added  to  the  mix,  where  their  actual  contribution  will  be  modulated  by  a  similarity  measure  between  the 
users  and  their  context  models. 

This  may  be  considered  as  a  first  step  towards  a  future  fully  functional  Intelligent  Assistant  /  Virtual  Analyst  capability 
(Gouin,  Lavigne  and  Bergeron-Guyard,  2012).  As  the  system  learns  from  user  interactions,  it  could  assemble  lines  of 
increasingly  complex  processing  steps  and  have  a  number  of  these  steps  performed  automatically. 

The  proposed  steps  may  also  involve  actions  assigned  to  other  team  members,  thus  supporting  a  collaborative  analysis 
process.  The  Workflow  Orchestrator  then  acts  both  as  the  tool  for  organizing  work  and  monitoring  its  execution, 
allowing  team  members  to  be  alerted  when  a  co-worker  has  completed  a  needed  task  or  when  another  person  is  waiting 
on  them  to  perform  an  action.  Boxes  for  disseminating  results  can  also  be  included  in  the  process  planning  so  that  this  is 
performed  automatically  as  soon  as  the  results  are  available. 

The  next  section  covers  collaboration  and  visualization  tools  that  are  of  interest  for  analysts  when  working  in 
collaboration  with  other  analysts. 

5.  SUPPORTING  COLLABORATION  AND  VISUALIZATION  TOOLS 

The  most  recent  technologies  allow  people  (e.g.  military  intelligence  analysts)  to  collaborate  with  each  other  and  to 
access,  visualize,  and  share  digitized  data  in  various  settings  such  as  wireless  hand-held  mobile  devices,  over  networked 
computers,  through  display  walls,  tabletop  surfaces,  to  name  just  a  few.  This  kind  of  collaboration  is  named  technology- 
mediated  collaboration.  Chosen  technologies  must  overcome  time  and  space  constraints  when  people  having  to 
collaborate  on  a  common  task  cannot  be  gathered  in  the  same  room  or  cannot  exchange  information  in  real-time.  Of 
course,  technology  must  also  facilitate  collaboration  between  people  localized  in  the  same  room  at  the  same  time. 

In  order  to  assist  military  intelligence  analysts  in  their  job  as  a  team,  technologies  must  allow  every  single  user  to  work 
and  perform  individual  tasks  (e.g.  annotation  in  the  form  of  labeling  or  tagging ,  interactive  visualization,  notification, 
distributed  visual  analytics...)  while  contributing  to  the  realization  of  greater  objectives  shared  by  the  team.  Figure  3 
exposes  a  non-exhaustive  summary  of  (combination  of)  technologies  based  on  the  time/space  categorization  matrix 
(Johansen,  1988)  that  considers  work  contexts  along  the  space  and  time  dimensions:  whether  collaboration  between  co¬ 
located  or  geographically  distributed  analysts  and  whether  they  collaborate  at  the  same  time  (synchronous)  or  not 
(asynchronous).  As  expressed  by  (Rodden  and  Blair,  1991),  in  the  day-to-day  activity,  a  whole  team  can  be  either  purely 
co-located,  virtually  co-located  (different  locations  but  connected  via  video  and/or  audio  systems),  locally  remote  (e.g.  in 
the  same  building),  worldwide  remote  (limited  accessibility  between  team  members)  or  simply  a  mix  of  the  four  ones. 
Furthermore,  the  time  can  be  as  well  purely  synchronous  or  asynchronous  as  a  mix  of  the  two  ones. 

Another  aspect  to  keep  in  mind  while  doing  collaborative  work  is  the  fact  that  tools  used  by  a  single  user  to  do  his/her 
job  have  usually  not  been  designed  and  developed  to  be  useful  and  usable  for  both  individual  and  collaborative  work 
(e.g.  a  shared  touch-table).  Consequently,  when  people  have  to  collaborate  together  on  such  a  tool,  part  of  the  work  has 
to  be  realized  in  a  sequential  and  iterative  way  by  team  members  or  the  interface  must  be  adapted  in  order  to  allow 
multiple  collaborators  to  interact  simultaneously.  From  time  to  time  context-aware  technologies  can  encompass  this 
issue,  especially  for  visualization  and  interactive  technologies. 

Virtual  collaboration  (Quarter  2  to  4,  Figure  3),  the  opposite  of  direct  collaboration  (Quarter  1,  Figure  3),  is  the  method 
of  collaboration  between  people  not  physically  present  in  the  same  location.  This  fully  technology-mediated 
collaboration  “follows  the  same  process  as  direct  collaboration,  but  the  parties  involved  in  virtual  collaboration  do  not 
physically  interact  and  communicate  exclusively  through  technological  channels”  (Peters  and  Manz,  2007).  Dispersed 


teams  use  virtual  collaboration  to  simulate  /  to  mimic  the  sharing  of  information  present  in  face-to-face  meetings, 
communicating  virtually  through  visual,  verbal,  written,  and  digital  means. 

Mobile  collaboration5  is  a  process  of  communication  using  electronic  assets  and  accompanying  software  designed  for 
use  in  remote  locations.  Recent  technological  advancements  in  mobile  collaboration  have  extended  the  capabilities  of 
traditional  videoconferencing  beyond  the  offices  and  meeting  rooms  for  use  with  hand-held  mobile  devices,  enabling 
collaboration  independent  of  location  (by  exploiting  wireless,  cellular  and  broadband  technologies)  and  permitting  true 
mobile  collaborative  possibilities.  Newest  generations  of  hand-held  electronic  devices  feature  video,  audio,  and  on¬ 
screen  drawing  in  addition  to  capabilities  to  broadcast  over  (secure)  networks,  enabling  multi-party  conferencing  in  real 
time. 

“Video  conferencing  has  become  an  umbrella  term  to  describe  all  types  of  video  based  communication  (video  chat 
between  two  or  more  people  from  laptops,  a  point  to  point  call  using  sophisticated  systems  found  in  conference  rooms, 
calls  between  two  phones  reminiscent  of  the  original  picture  phone  or  multi-party  calls  similar  to  audio  conferencing.”6 
Telepresence  refers  to  technologies  which  allow  people  to  feel  as  if  they  were  present,  to  give  the  appearance  of  being 
present,  or  to  have  an  effect  at  a  place  other  than  their  real  location  (e.g.  telerobotics).  A  popular  application  is  found  in 
telepresence  videoconferencing.  Telepresence  via  video  deploys  greater  technical  sophistication  and  improved  fidelity  of 
both  sight  and  sound  than  in  traditional  videoconferencing.  Holographic  telepresence7  is  an  evolving  technology  for  full- 
motion,  three-dimensional  video  conferencing.  Holography8  is  a  technique  which  enables  three-dimensional  images 
(holograms)  to  be  made. 


Co-located  Remote  /  Distributed 


Quarter  2 

Quarter  3 

Quarter  1 

Quarter  4 

Figure  3.  Collaborative  visualization  in  terms  of  time  and  space  aspects  and  types  of  technology  involved 

A  powerful  collaboration  capability  includes  the  visualization  aspect  which  is  a  means  to  achieve  effective  human 
communication.  Vision  is  the  humans’  dominant  sense.  More  than  50%  of  a  human  brain’s  neurons  are  used  to  see 


5  http://en.wikipedia.org/wiki/Mobile  collaboration 

6  http://bluejeans.com/blog/video-conferencing-vs-telepresence 

7  http://whatis.techtarget.com/defmition/holographic-telepresence 

8  http://www.sciencedaily.com/releases/20 1 5/02/150204090 101  .htm 


(including  3D  and  full  color).  Multi-dimensional  visualization  in  an  immersive  display  system  (e.g.  3D  image  display 
wall,  CAVE  Automatic  Virtual  Environment  -  CAVE9,  Head  Mounted  Display  -  HMD...)  is  also  to  be  considered  in 
our  digitized  world  because  these  systems  support,  among  other  things,  the  ability  to  (1)  simultaneously  exhibit  a  large 
amount  of  data;  (2)  represent  multidimensional  dataset  more  efficiently  compared  to  any  other  display  devices;  (3) 
explore  data  in  a  more  intuitive  way  through  interactions  with  visually  rendered  data;  (4)  analyze  and  interpret  data 
faster;  and  (5)  help  escaping  the  conventional  bias  towards  2D  computing  by  organizing  content  more  effectively  in  3D. 
Interactive  visualization  strengthens  collaboration  for  the  reason  that  this  approach  is  conducive  to  incrementally 
building  a  mental  assessment  of  the  data  meaning. 

In  order  to  take  efficiently  advantage  of  these  visualization  and  collaboration  technologies  to  enhance  the  intelligence 
analysis  process  and  provide  adaptive  user  interfaces,  the  Context  Manager  has  to  consider  multiple  aspects  of  the 
context.  The  Workflow  Orchestrator  should  support  each  user  during  the  execution  of  his/her  work  by  suggesting  the 
most  appropriate  (available)  interaction  devices  and  corresponding  Interactive  Visualization  Widgets  for  his/her  current 
task  (e.g.  interactive  displays  and  tablets  could  be  adequate  devices  for  the  Workflow  Exploration  widget;  shared  screens 
can  be  used  to  disseminate  information  and  results  to  team  members).  The  Workflow  Orchestrator  must  also  ensure  that 
display  and  interactions  devices  are  exploited  efficiently  by  the  whole  team,  especially  when  resources  are  limited  (e.g. 
manage  displayed  data  on  shared  display  to  provide  relevant  information  for  the  whole  team).  Figure  4  gives  examples 
of  collaboration  and  visualization  technologies  of  interest  that  were  explored  within  DRDC  research  projects.  The  way 
they  could  be  exploited  with  Interactive  Visualization  Widgets  and  other  application  tools  to  perform  a  number  of 
collaborative  military  intelligence  tasks  is  also  discussed. 


9  The  CAVE  (from  Mechdyne  -  http://www.mechdyne.com/)  at  DRDC-Valcartier  is  a  multiperson,  four-sided  (three  walls  -  two  are 
pivoting  around  an  axe  from  0  to  90°  -  and  a  floor),  high-resolution  3D  environment  that  is  used  for  viewing  and  interacting  with 
virtual  content.  To  achieve  realistic  interactions  with  immersive  displays  the  tracking  system  (IS-900  system  from  Intersense  - 
http://www.intersense.com)  has  fast  update  rates,  low  latency  and  smooth  tracking.  Consequently,  it  provides  smooth  and  precise 
position  and  orientation  (6-DOF)  of  the  user’s  head  while  not  interfering  with  the  user’s  immersive  experience.  The  tracking  system 
offers  ergonomically  designed  devices:  a  head  tracker  attached  to  stereo  glasses  (CrystalEyes®3  from  Stereo  Graphics  Corp.  - 
http://www.reald.com)  and  a  tracked  wand  (from  Inter  Sense)  which  incorporates  buttons  and  a  joystick  for  interaction  with  the  virtual 
content.  Stereoscopic  rendering  based  on  stereo  (active)  glasses  provides  to  the  user  the  depth  perception  of  virtual  content. 


Direct  Collaboration  (co-located  +  synchronous) 


Using  an 

immersive  display 
system  - 

Collaborative  data 
analysis  and 
interactive  data 
visualization. 


Using  a  tabletop 
display  (e.g. 
Interactive 
Visualization 
Widgets ,  Workflow 
Orchestrator, 
Workflow  Tracer). 


Direct  Collaboration  -  People  working 
collaboratively  in  the  same  virtual  environment ,  one 
person  working  directly  with  the  immersive  version 
(left  image)  and  the  others  co-located  in  the  same 
space  working  with  the  desktop  version  (left  image). 

Remote  Collaboration  -  Idem  as  previous  one  but 
additional  team  members  work  in  remote  (right 
image).  No  videoconferencing  or  telepresence,  the 
virtual  environment  is  the  shared  space  and 
everyone  does  what  he/she  has  to  do  to  realize  the 
shared  goal. 


Direct  Collaboration  -  Through  an  interactive  large  display  wall  (e.g.  enhancing 
Visual  Analytics  widgets  such  as  the  Graph  Analyzer  widget  designed  for  Social 
Network  Analysis  (Lavigne,  Lecocq,  Martineau  and  Mokhtari,  2014)). 

Remote  Collaboration  -  A  display  wall  could  also  be  used  when  analysts  are  in 
remote  locations  (it  becomes  a  shared  surface  between  the  dispersed 
collaborators). 

Continuous  Task  -  A  display  wall  could  support  ongoing  work  between 
co-located  collaborators. 


Remote  Collaboration  -  Same  immersive 
virtual  environment  shared  by  two  different 
locations.  Analysts  manipulate  tools  and 
data  in  order  to  augment  their  problem 
comprehension. 


Figure  4.  Examples  of  technologies  of  interest  for  collaborative  military  intelligence  analysis 


6.  CONCLUSION 

During  the  last  few  years,  DRDC  has  investigated  methods  and  techniques  to  help  military  intelligence  analysts  to  do 
their  job.  Widget  prototypes  related  to  Social  Network  Analysis  (through  different  use-cases),  Visual  Analytics  (in  the 
Maritime  domain),  Case-Based  Reasoning ,  to  name  just  these  ones,  have  been  developed  and  integrated  in  the  current 
version  of  the  S3  prototype.  DRDC  is  currently  investigating  new  capabilities  and  also  the  technological  means  through 
which  they  can  be  deployed  to  be  of  better  interest. 

The  proposed  collaborative  military  intelligence  production  capability  is  a  context-aware  system  adapting  its  behavior 
and  features  dynamically  according  to  various  aspects  of  the  context  in  order  to  optimize  exploitation  of  the  system  by 
multiple  analysts  simultaneously  while  providing  continual  system  feedback  and  a  seamless  user  experience  across  the 
whole  system.  The  Context  Manager  sub-system  is  responsible  for  context  model  persistence  and  has  to  manage 


system’s  behaviors  depending  on  the  current  situation.  Context  information  stored  in  the  model  can  be  partitioned  in  four 
categories:  (1)  operational,  (2)  user,  (3)  resources  and  collaboration  and  (4)  extrinsic  contexts. 

Contextual  information  can  be  exploited  to  adapt  three  aspects  of  the  user  interface:  (1)  content,  (2)  presentation  and  (3) 
proposed  actions.  Part  of  the  Context  Manager,  the  Workflow  Orchestrator  has  to  consider  this  contextual  information  in 
order  to  plan  and  coordinate  the  concurrent  executions  of  various  processing  and  collaboration  tasks.  The  Workflow 
Orchestrator  also  extends  the  Workflow  Tracer ,  keeping  tracks  of  every  step  performed  by  the  analyst,  by  exploiting 
usage  history  statistics  to  provide  appropriate  action  recommendations. 

A  wide  range  of  visualization  and  interaction  technologies  are  available  to  implement  a  collaborative  intelligence 
production  capability.  Depending  on  deployment  setup,  visualization  and  interaction  needs,  the  chosen  technologies  can 
take  many  forms,  from  the  traditional  laptop  computers  to  fully  immersive  3D  environments.  Moreover,  shared  displays 
and  devices  with  specific  interaction  features  (e.g.:  touch- wall,  CAVE)  could  be  unique  or  limited  in  the  working 
environment  and  must  be  used  efficiently  by  the  team.  Consequently,  the  Context  Manager  must  not  only  consider 
visualization  and  interaction  features  of  available  devices  but  it  must  also  take  into  considerations  the  availability  and 
proper  usage  of  those  limited  assets. 


Figure  5.  Implementation  of  part  of  the  FIAC  vision  in  a  virtual  immersive  context 
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